ESTIMATES OF AUTOMORPHIC FUNCTIONS 

JOSEPH BERNSTEIN AND ANDRE REZNIKOV 



Abstract. We present a new method of estimating trilinear period for automorphic 
representations of SI^R). The method is based on the uniqueness principle in rep- 
resentation theory. We show how to separate the exponentially decaying factor in the 
triple period from the essential automorphic factor which behaves polynomially. We also 
describe a general method which gives an estimate on the average of the automorphic 
factor and thus prove a convexity bound for the triple period. 



1. Introduction 

1.1. Maass forms. Let Y be a compact Riemann surface with a Riemannian metric 
of constant curvature —1 and the associated volume element dv. The corresponding 
Laplace-Beltrami operator is non-negative and has purely discrete spectrum on the space 
L 2 (Y, dv) of functions on Y. We will denote by = [Iq < jii < < ... its eigenvalues 
and by fa = Mi the corresponding eigenf unctions (normalized to have L 2 norm one). In 
the theory of automorphic forms the functions Mi are called automorphic functions or 
Maass forms (after H. Maass, |Mj). 

The study of Maass forms plays an important role in analytic number theory. 

We are interested in their analytic properties and will present a new method of bounding 
some important quantities arising from fa. 

1.2. Triple products. For any three Maass forms fa, <pj, <frk we define the following 
triple product or triple period: 



(1) djk = / fafafadv . 



Y 



We would like to bound the coefficient c^ fc as a function of eigenvalues /Xj, /ij, In 
particular, we would like to find bounds for these coefficients when one or more of these 
indices tend to infinity. 



1.3. Motivation. First of all we would like to explain why this problem is interesting. 

The explanation goes back to pioneering works of Rankin and Selberg (see (Raj, |5e]). 

They discovered that in special cases triple products as above give rise to automorphic 

L-functions. That allowed them to obtain analytic continuation and effective bounds for 

l 
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these L-functions and as an application to obtain bounds on Fourier coefficients of cusp 
forms towards Ramanujan conjecture. 

Since then the Rankin- Selberg method has had many generalizations. Recently, for Y 
arising from the full modular group SL 2 (Z) and for cuspidal functions 4>, Watson (see 
[WaJ) proved the following beautiful formula: 



(2) 



Y 



G(Xi, Xj, Xk) 



L(l/2, 



L(l, 0j, Ad)L(l, 4>j, Ad)L(l, (f> k , Ad) 



Here X t is a natural parameter of an eigenfunction t related to the eigenvalue by fi t = 

1— A 2 

—^ L . The unctions L(s,(pi ® <pj ® <f>k) and L(s, <p t , Ad) are appropriate automorphic L- 
functions associated to <pi, and the function G(Xi, Xj, Xk) is an explicit rational expression 
in the ordinary T-functions. The relation @ can be viewed as a far reaching generalization 
of the original Rankin- Selberg formula. It was motivated by a work |HK| by Harris and 
Kudla on a conjecture of Jacquet. 



1.4. Results. In this paper we will consider the following problem. We fix two Maass 
forms (p = (p T , <p' = (f) T i as above and consider coefficients defined by the triple period as 
above: 

(3) Ci = J (f)(f>'(f>idv 
as {(pi = 0Ai} run over the basis of Maass forms. 

Thus we see from (J2J) that the estimates of coefficients q are equivalent to the estimates of 
the corresponding L-functions. One would like to have a general method of estimating the 
coefficients q and similar quantities. This problem was raised by Selberg in his celebrated 
paper [5ej . 

Let us understand what kind of bounds on the left hand side of (0) one would like to have 
in order to estimate effectively L-functions involved in the right hand side of (j2J) (or at 
least the ratio of L-functions). 

We note first that one expects that q have exponential decay in |Aj| as i goes to oo. 
Namely, general experience from the analytic theory of automorphic L-functions tells us 
that L-functions have at most polynomial growth when |Aj| — > oo. Hence, analyzing the 
function G(X), one would expect from (j2J) and the Stirling formula for the asymptotics of 
T-function that the normalized coefficients 

( 4 ) h = |c i | 2 exp(||A i |) 

have at most polynomial growth in |Aj|, and hence q decay exponentially. However, it is 
difficult to see from the definition of the coefficients q that they have exponential decay 
and it is not clear what should be the rate of this decay. 
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The fact that an exponential decay with the exponent | holds for a general Riemann 
surface was first shown by Good and Sarnak (see |Goj and |Salj ). Both proofs used 
ingenious analytic continuation of automorphic functions in the variable parameter. 

In this paper we will explain how to naturally separate the exponential decay from a poly- 
nomial growth in coefficients Cj using representation theory. We also prove the following 

Theorem. There exists an effectively computable constant A such that the following 
bound holds for arbitrary T > 

(5) J2 b ^ A - 

T<\\ t \<2T 

1.5. A conjecture. The estimate in the theorem is tight but if we try to use it to get a 
bound for an individual term hi we get only an inequality 

(6) h < A . 

According to Weyl's law there are approximately cT 2 eigenvalues /ij with Aj between T 
and 2T, so the individual bound for the coefficient bi is definitely not tight. We would 
like to make the following conjecture concerning the size of coefficients b{. 

Conjecture. For any e > there exists a constant C £ > such that 

bi < C e |Ai| 2+6 , 

as | Aj| —y oo. 

For Y arising from congruence subgroups this conjecture is consistent with the Lindelof 
conjecture for appropriate automorphic L-functions (see |BKI]. [Sa2j and |Waj for more 
details). We note that the bound in the Theorem above corresponds to the so-called 
convexity bound. 

1.6. The method. The first proof of the (slightly weaker) version of Theorem 11.41 ap- 
peared in [BRlJ. It was based on the analytic continuation of representations from a real 
group to a complex group (generalizing methods of |Salj ). The method based on the 
analytic continuation was extended in |KSj to the case of higher rank groups. While it 
gives bounds which are tight for general representations, it was not able, so far, to cover 
cases relevant to L-functions. 

The proof we present here is based on the uniqueness of triple product in representation 
theory. It has an advantage that it could be generalized to higher rank groups and gives 
bounds which are consistent with the theory of L-functions. The present method also 
could be applied to p-adic groups (unlike methods of |BRlj ). 

We describe now the general ideas behind our new proof. It is based on ideas from 
representation theory. Namely, we use the fact that every automorphic form generates 
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an automorphic representation of the group G = PGL 2 (IR); this means that starting from 
we produce a smooth irreducible representation of the group G in a space V and its 
realization v : V — > C°°(X) in the space of smooth functions on the automorphic space 
X = T\G. 

The triple product q = j Y cfxp' <pidv extends to a G-equivariant trilinear form on the 
corresponding automorphic representations l aut : V Cg) V (g) Vi — > C, where V = V T , V = 

v T ,,v i = v Xi . 

Then we use a general result from representation theory that such G-equivariant trilinear 
form is unique up to a scalar. This implies that the automorphic form l aut is propor- 
tional to an explicit "model" form l mod which we describe using explicit realizations of 
representations of the group G; it is important that this last form carries no arithmetic 
information. 

Thus we can write l aut = ai ■ l mod for some constant Oj and hence c$ = l aut (e T <S> e T i ® e^) = 
a . . jmod^ ^ e ^_ ; (gj e A J, where e T , e T /, e\ i are K- invariant unit vectors in the automorphic 
representations V, V, Vi corresponding to the automorphic forms <j), <j)' and 0j. 

It turns out that the proportionality coefficient a, in the last formula carries an important 
"automorphic" information while the second factor carries no arithmetic information and 
can be computed in terms of T-functions using explicit realizations of representations V T , 
V T i and V\ t . This second factor is responsible for the exponential decay, while the first 
factor tjj has a polynomial behavior in parameter Aj. 

In order to bound the quantities a^, we use the fact that they appear as coefficients in 
the spectral decomposition of the diagonal Hermitian form on the space E = V T ® V T i 
(see I4.2[ I4.3J1 . This gives an inequality \ a i\ 2 Hi < where Hi is an Hermitian form 
on E induced by the model trilinear form l mod : V <S> V <S> Vi — > C as above. 

Using the geometric properties of the diagonal form and simple explicit estimates of forms 
Hi we establish the convexity bound for the coefficients ctj. 

It is known that the uniqueness principle plays a central role in the theory of automorphic 
functions (see |PSj ). The impact that the uniqueness has on the analytic behavior of 
automorphic functions is yet another manifestation of this principle. 
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referee for helpful comments. 
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Israel Science Foundation, the Emmy Noether Institute for Mathematics (the Center of 
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The paper was written during our stay at IHES and is an extended version of the talk 
given by the first author at Cartier Colloquium in June 2002. 



2. Representation theoretic setting 
We recall the standard connection of the above setting with representation theory (see 

my 

2.1. Automorphic functions and automorphic representations. Let us describe 
the geometric construction which allows one to pass from analysis on a Riemann surface 
to representation theory. 

Let H be the upper half plane with the hyperbolic metric of constant curvature — 1. 
The group ST^M) acts on EI by fractional linear transformations. This action allows to 
identify the group PSLi2(M.) with the group of all orientation preserving motions of HI. For 
reasons explained bellow we would like to work with the group G of all motions of EI; this 
group is isomorphic to PGL 2 (M.). Hence throughout the paper we denote G = PGL 2 (M^). 

Let us fix a discrete co-compact subgroup T C G and set Y = T \ HI. We consider the 
Laplace operator on the Riemann surface Y and denote by /Zj its eigenvalues and by (pi 
the corresponding normalized eigenfunctions. 

The case when T acts freely on HI precisely corresponds to the case discussed in the 
introduction (this follows from the uniformization theorem for the Riemann surface Y). 
Our results hold for general co-compact subgroup T (and in fact, with slight modifications, 
for any lattice FcG). 

We will identify the upper half plane HI with G/K, where K = PO(2) is a maximal 
compact subgroup of G ( this follows from the fact that G acts transitively on HI and the 
stabilizer in G of the point Zq = i G HI coincides with K). 

We denote by X the compact quotient V \ G (we call it the automorphic space). In the 
case when V acts freely on HI one can identify the space X with the bundle of unit tangent 
vectors to the Riemann surface Y — T \ EL 

The group G acts on X (from the right) and hence on the space of functions on X. We 
fix the unique G-invariant measure \Lx on X of total mass one. Let L 2 (X) = L 2 (X, dfix) 
be the space of square integrable functions and (Tlx, G, L 2 (X)) the corresponding unitary 
representation. We will denote by Px the Hermitian form on L 2 (X) given by the scalar 
product. We denote by || ||x or simply |j j| the corresponding norm and by (f,g)x the 
corresponding scalar product. 

The identification Y = T \ U ~ X/K induces the embedding L 2 (Y) C L 2 (X). We will 
always identify the space L 2 (Y) with the subspace of i^-invariant functions in L 2 (X). 
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Let be a normalized eigenfunction of the Laplace-Beltrami operator on Y. Consider 
a closed G-invariant subspace C L 2 (X) generated by under the action of G. It is 
well-known that (tt, L) = (tt^, L^) is an irreducible unitary representation of G (see |G6j ). 

Usually it is more convenient to work with the space V = L°° of smooth vectors in L. 
The unitary Hermitian form Px on V is G-invariant. 

A smooth representation (tt, G, V) equipped with a positive G-invariant Hermitian form 
P we will call a smooth pre-unitary representation; this simply means that V is the 
space of smooth vectors in the unitary representation obtained from V by completion 
with respect to P. 

Thus starting with an automorphic function we constructed an irreducible smooth pre- 
unitary representation (tt, V) . In fact we constructed this space together with a canonical 
morphism v : V — > C°°(X) since C°°(X) is the smooth part of L 2 (X). 

Definition. A smooth pre-unitary representation (tt, G, V) equipped with a G-morphism 
v : V — > C°°(X) we will call an X-enhanced representation. 

In this note we will assume that the morphism v is normalized, i.e. it carries the standard 
L 2 Hermitian form Px on C°°(X) into Hermitian form P on V . 

Thus starting with an automorphic function cf) we constructed 

(i) An X-enhanced irreducible pre-unitary representation (it, V, u), 

(ii) A K- invariant unit vector ey G V (this vector is just our function 0). 

Conversely, suppose we are given an irreducible smooth pre-unitary X-enhanced repre- 
sentation (tt, V, v) of the group G and a ii'-fixed unit vector ey £ V. Then the function 
= v(ey) G G°°(X) is K- invariant and hence can be considered as a function on Y . 
The fact that the representation (tt, V) is irreducible implies that <fi is an automorphic 
function. 

Thus we have established a natural correspondence between Maass forms <fi and tuples 
(it, V, v, ty), where (it, V, v) is an X-enhanced irreducible smooth pre-unitary representa- 
tion and ey G V is a unit X-invariant vector. 

2.2. Decomposition of the representation (H_x, G, L 2 (X)). It is well known that in 
case when X is compact the representation (Ux,G, L 2 (X)) decomposes into a direct 
(infinite) sum 

(7) L 2 (X) = ® 3 (tt 3 ,L 3 ) 

of irreducible unitary representations of G (all representations appear with finite multi- 
plicities (see |G6j )). Let (tt,L) be one of these irreducible "automorphic" representations 
and V = L°° its smooth part. By definition V is given with a G-equivariant isometric 
morphism v : V — > G°°(X), i.e. V is an X-enhanced representation. 
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If V has a A-invariant vector it corresponds to a Maass form. There are other spaces 
in this decomposition which correspond to discrete series representations. Since they are 
not related to Maass forms we will not study them in more detail. 

2.3. Representations of PGZ^IR). All irreducible unitary representations of G are 
classified. For simplicity we consider those with a nonzero K-fixed vector (so called 
representations of class one) since only these representations arise from Maass forms. 
These are the representations of the principal and the complementary series and the 
trivial representation. 

We will use the following standard explicit model for irreducible smooth representations 

of a 

For every complex number A consider the space V\ of smooth even homogeneous functions 
on M 2 \ of homogeneous degree A — 1 (which means that f(ax,ay) = \a\ x ~ 1 f(x,y) for 
all a G R \ 0). The representation (itx, V\) is induced by the action of the group GL 2 (M.) 
given by it x (g) / \x , y) = /(g _1 (x, y)) \ det g]^ 1 ^ 2 . This action is trivial on the center of 
GL 2 {R) and hence defines a representation of G. The representation (tt x , V\) is called 
representation of the generalized principal series. 

When A = it is purely imaginary the representation (tcx, V\) is pre-unitary; the G- invariant 
scalar product in V\ is given by (/, g),^ = ^- J sl fgdO. These representations are called 
representations of the principal series. 

When A G (—1,1) the representation (7T\,V\) is called a representation of the comple- 
mentary series. These representations are also pre-unitary, but the formula for the scalar 
product is more complicated (see jG6j ). 

All these representations have A-invariant vectors. We fix a A-invariant unit vector 
eA G Va to be a function which is one on the unit circle in IR 2 . 

Representations of the principal and the complimentary series exhaust all nontrivial irre- 
ducible pre-unitary representations of G of class one. 

In what follows we will do necessary computations for representation of the principal series. 
Computations for the complementary series are a little more involved but essentially the 
same (compare with |BRlj . section 5.5, where similar computations are described in 
detail). 

Suppose we are given a class one A-enhanced representation v : V\ — > C°°(A); we assume 
v to be an isometric embedding. Such v gives rise to an eigenfunction of the Laplacian 
on the Riemann surface Y = X/ K as before. Namely, if e\ G V\ is a unit A-fixed vector 
then the function = z/(ca) is a normalized eigenfunction of the Laplacian on the space 
Y = X/K with the eigenvalue jj, = This explains why A is a natural parameter to 

describe Maass forms. 



2.4. Triple products. We introduce now our main tool. 
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2.4.1. Automorphic triple products 
tions of G 

We define the G-invariant trilinear 

(8) Ck,7r 3 (^i ® v 2 ® v 3 ) = / (f> vi (x)(f> va (x)<f> va {x)dfix , 

Jx 

where = Uj(vj) G C7°°(X) for G V,-. 

In particular, the triple periods c, in (JBJ) can be expressed in terms of this form as Cj = 
^.(er ® e r ' ® e^J, where eA G V\ is the i^-fixed unit vector. 

2.4.2. Uniqueness of triple products. The central fact about invariant trilinear functionals 
is the following uniqueness result: 

Theorem. Let (7Cj,Vj), j = 1,2,3 , be three irreducible smooth admissible representa- 
tions ofG. Then dimHom G (^i ® V 2 <g> V 3 , C) < 1. 

Remark. The uniqueness statement was proven by Oksak in [0] for the group SL(2, C) 
and the proof could be adopted for PGL 2 {M) as well (see also |Moj and |Loj). For the 
p-adic GL(2) more refined results were obtained by Prasad (see [Fj). He also proved the 
uniqueness when at least one representation is a discrete series representation of GL 2 (M.). 

There is no uniqueness of trilinear functionals for representations of SL 2 (M>) (the space is 
two-dimensional). This is the reason why we prefer to work with PGL 2 (R). 

For SL 2 {R) one has the following uniqueness statement instead. Let (ir, V) and (a, W) 
be two irreducible smooth pre-unitary representations of SL 2 (M.) of class one. Then the 
space of SX 2 (M)-invariant trilinear functionals on V <E> V <8> W which are symmetric in the 
first two variables is one-dimensional. This is the correct uniqueness result needed if one 
wants to work with SL 2 {M) ; this was implicitly done in |Re2j . where the second author 
missed the absence of the uniqueness for SL 2 (M.). We take an opportunity to correct this 
gap. 

We note however, that the absence of uniqueness does not pose any problem for the 
method we present. All what is really needed for our method is the fact that the space 
of invariant functionals is finite dimensional . 

3. Triple products: exponential decay 

We now explain our method how to bound coefficients q. It is based on the uniqueness 
of trilinear functionals. 



Suppose we are given three X-enhanced representa- 
+ C°°PO, j = 1,2,3. 

form /^ i 7r2 : V x <g> V 2 <g> V 3 — >• C , by formula 



AUTOMORPHIC FUNCTIONS 



9 



3.1. Model triple products. Let (tt,V) and (71"', V) be automorphic representations 
corresponding to Maass forms and 0'. Any Maass form <pi gives us an automorphic 
representation , VaJ and hence defines a trilinear functional 

l^^:V®V'®V Xi ^C. 

In 15.11 we use an explicit model for representations tti, 712, ^3 to construct a model 
invariant trilinear functional which is given by an explicit formula. We call it the model 
triple product and denote by 1™°% 27T3 . 

By the uniqueness principle for representations ir, ir', 7Tj there exists a constant cij = a^y^ 
such that: 



/q\ iaut imod 

\ J ) 7r,7r',7Tj ~~ a i ' '-n-.Tr'TTi 



3.2. Exponential decay. This gives a formula for the triple products q 

(10) Q = Iff (e r g> e T , <g> e Ai ) = a* • /^ od (e r ® e T , ® e A J . 

Here we denoted l\ ut = V^l, „. , Z?"** = and ex is the unit i\"-fixed vector in the 

representation V^. 

The model triple product l™° d (e T <8> e r / ® e A J constructed in 15.11 is given by an explicit 
integral. In Appendix El we evaluate this integral by a direct computation in the model. 
It turns out that it has an exponential decay in | A | which explains the exponential decay 
of coefficients q. Namely, we prove the following 

Proposition. Set k\ := \l™ od (e T ® e T > <8> e A )| 2 - Then there exists a constant c > such 
that 

fc A = cexp(-||A|)-|A|- 2 (l + 0(|A|- 1 )) 

as |A| — > 00 and A G iM. 



4. Triple products: polynomial bounds 

We explain now how to obtain bounds on the coefficients a, (note that these coefficients 
encode deep arithmetic information - values of L-functions). 

Our method is based on the fact that these coefficients appear in the spectral decompo- 
sition of some geometrically defined Hermitian form on the space E which is essentially 
the tensor product of spaces V and V . 

More precisely, denote by L and V the Hilbert completions of spaces V and V, consider 
the unitary representation (II, G x G , L <g> L') of the group G x G and denote by E its 
smooth part; so E is a smooth completion of V <S> V. 

Denote by 7~C{E) the (real) vector space of continuous Hermitian forms on E and by 
7i + (E) the cone of nonnegative Hermitian forms. 
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We will describe several classes of Hermitian forms on E; some of them have spectral 
description, others are described geometrically. 

4.1. Hermitian forms corresponding to trilinear functionals. Let W be a smooth 
pre-unitary admissible representation of G. Any G-invariant functional I : V®V'®W — > C 
defines a G-intertwining morphism T : V <8) V' — > W* which extends to a G-morphism 

(11) T l :E^W. 

where we identify the complex conjugate space W with the smooth part of the space W*. 

The standard Hermitian form (scalar product) Pw on the space W induces the Hermitian 
form P on W. Using the operator T l we define the Hermitian form H l on the space E by 
H i = (T l )*(P), i.e. H\u) = P{T\u)) for u G E. 

We note that if the representation of G in the space W is irreducible then starting with the 
Hermitian form H l we can reconstruct the space W, the functional / and the morphism 
T l uniquely up to an isomorphism. 

Let us introduce a special notation for the particular case we are interested in. For 
any number A 6 it consider the representation of the principal series W = V\, choose 
the model trilinear functional l mod : V ® V (g) V\ — *■ C described in 15.11 and denote the 
corresponding Hermitian form on E by H™ od . 

4.2. Diagonal form Consider the space C°°(X x X). The diagonal A : X — > 
X x X gives rise to the restriction morphism : C°°(X x X) — > C°°(X). We define a 
nonnegative Hermitian form on C°°(X x X) by Ha = (^a)*(-Px), i-e. 

H A (u) = P x (r A (u)) = j x \r A (u)\ 2 dfi x for u G C°°{X x X). 

We call H A the diagonal form. 

More generally, if L is a closed subspace of L 2 (X) and prL '■ L 2 (X) —>■ L the orthogonal 
projection onto L we can define a Hermitian form Pi on C°°(X) by Pl = (prL)*{Px) and 
consider the induced Hermitian form Hl = (ta)*(Pl) on G°°(X x X). 

Clearly the correspondence L \— > Hl is additive (which means that Hl+l' = Hl + Hu if 
L and V are orthogonal) and monotone. 

4.3. First basic inequality. Let us realize the space E = V®V'a,s&Gx G-invariant 
subspace of G°°(X x X). We consider the restrictions of the Hermitian forms Ha,Hl 
discussed above to the space E and will denote them by the same symbols. 

Claim. Let (f>\. be a Maass form. Consider the G -invariant subspace Li C L 2 (X) 
generated by 4>\ i and its complex conjugate Li C L 2 (X). 

Then on the space E the Hermitian form H L _ coincides with the form H^ 1 corresponding 
to the automorphic trilinear form I = 1% U *, : V <8> V <8> V\ t — > C. 
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Indeed, if we identify the space Lj with L* then the operator pri. o r A : E —>■ Li coincides 
with the operator T l corresponding to the automorphic trilinear form I = 1® U *, n .. 

This claim implies the first basic inequality 



(12) j2\ a ^ H r d < H ^- 

Indeed, by the uniqueness principle (JHJ) we have: 



(13) Hlf = \ ai \ 2 ■ H™ d , 

where aj = a n y t7Ti are as in (jHJ). 

Since all the spaces Li are orthogonal we have Yli H® ut < H A which proves the first basic 
inequality. 



4.4. Second basic inequality. We would like to use the inequality (fT2*|) to bound the 
coefficients aj. In order to do this we have to establish some bounds for the diagonal form 
H A . 

The group G x G naturally acts on the space of Hermitian forms on C°°(X x X) - we 
denote this action by II. We extend this action to the action of the algebra H(G x 
G) = C^°(G x G, M) of smooth real valued functions with compact support. Note that 
if h G H(G x G) is a nonnegative function then the operator H(h) preserves the cone of 
positive forms. 

We have then the second basic inequality 

Claim. Let h e H(G x G) be a non-negative function. Then there exists a constant C, 
depending on h, such that we have H{h)H& < C ■ Pxxx, where Pxxx is the standard L 2 
Hermitian form on the space C°°(X x X). 

Proof. Let u E C°°(X x X) and / = \u\ 2 . Then P X xx(u) = (fi, f) and U(h)H A (u) = 
(/i', /), where \i = fixxx and // = II(/i)(A*(/ix)) are two measures on X x X. 

Since the measure /i' is smooth it is bounded by Cfi. □ 



Note that the bound in the claim is essentially tight. Namely if the function h has large 
enough support then we also have a bound in the opposite direction. 



4.5. Positive functionals. We can now prove that the coefficients have at most poly- 
nomial growth in |Aj|. 
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We start with the inequality (J12j) of non-negative forms. We want to produce out of it 
an inequality for coefficients ctj. There is a standard way to do this by means of positive 
functionals on the space of Hermitian forms TC(E). 

Definition. A positive functional on the space TC(E) is an additive map p : 7i{E) + — > 

It is easy to see that the positive functional p is automatically monotone and homogeneous 
(i.e. p(H) < p(H') ilH <H' and p{aH) = ap(H) for a > 0). 

Example. Any vector u e E gives us an elementary positive functional p u defined by 
p u {H) = H{u). 

Fix a positive functional p and consider the weight function h(X) = p(H™ od ). Then from 
the first basic inequality fl"2|) we can deduce the following inequality for a weighted sum 
of coefficients |a, | 2 : 



$>(A,)H 2 <p(tf A ) • 

4.6. Test functional pt- For any real T we construct in !5.2l the positive "test" functional 
Pt on 7~t(E) with the properties described in the proposition below. Let us fix automorphic 
representations V, V C C°°(X), E = V <g) V C C°°(X x X) as above. 

Proposition. We can find a constant C which depends only on G and T and a constant 
T which depends on V and V such that for any T > T there exists a positive functional 
Pt on 7~C(E) satisfying 

(14) p T (H A ) < CT 2 , 

(15) hr{\) := p T (H™ d ) > 1 for any |A| < 2T . 



4.7. Proof of Theorem II. 4L Consider the inequality \ai\ 2 pr(H™° d ) < pt(-^a)- 

The right hand side Pt(-^a) is bounded by CT 2 . In the left hand side we can leave only 
terms with |Aj| < 2T. Thus we arrive at inequality 



(16) Yl H 2 < ct2 - 

\Xi\<2T 

This gives the desired bound for |aj| 2 . 

According to Proposition IH.2I there exists a constant b such that b{r 2 < b\ai\ 2 for T < 
W\ — 2T. This shows that Y1t<\\ |<2T^* — ^ ^ or some constant A, which finishes the 
proof of Theorem 11.41 
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4.7.1. A conjecture. One can show (see |Relj ) that the mean-value result in (J 16)) is essen- 
tially sharp. One expects that for T < |Aj| < 2T all terms in the sum are at most of 
order T £ for any e > 0. Hence, we have established a sharp bound on the average and a 
rather weak bound for each term. This is a typical situation which one often encounters 
in the analytic theory of L-functions, the so-called convexity bound. The major problem 
hence is to find a method which would allow us to obtain better bound for a single term 
or for a short interval - the so-called subconvexity bounds. 

We would like to make the following conjecture concerning the size of coefficients a 7T y t7Ti 
which is equivalent to Conjecture 11.51 

Conjecture. For fixed it, it' and for any e > there exists C e > independent of Aj 
such that 

k^7r,7r',7r\ . ^ C £ Aj , 

as | Aj | — > oo. 

5. Construction of model trilinear functionals and of test functionals 

5.1. Model trilinear functionals. For every A G C we denote by (tt\,V\) the smooth 
class one representation of the generalized principle series of the group G = PGL2(M.) 
described in 12 .31 We will use the realization of (tt\, V\) in the space of smooth homogeneous 
functions on R 2 \ of homogeneous degree A — 1 . 

For explicit computations it is often convenient to pass from plane model to a circle 
model. Namely, the restriction of functions in V\ to the unit circle S 1 C M 2 defines an 
isomorphism of the space V\ with the space C°°(S x ) even Q f even smooth functions on S* 1 
so we can think about vectors in V\ as functions on S 1 . 

In this section we describe the model invariant trilinear functional using the geometric 
models. Namely for given three complex numbers Aj, j = 1,2,3, we construct explicitly 
nontrivial trilinear functional l mod : V\ 1 <S> V\ 2 ® V\ 3 — » C by means of its kernel. 

5.1.1. Kernel of l mod . Let u)(£,rj) = £1772 — £2^1 be S^CR) -invariant of a pair of vectors 
f , 77 G M 2 . We set 

(17) K Xl , X2>X3 ( Sl ,s 2 ,s 3 ) = K S2 ,s 3 )| (a - 1)/2 Hsus^-^uisu^- 1 ^ 

for si, s 2 , S3 6 K 2 \ 0, where a = Ai - A 2 - A3, (3 = — Ai + A 2 - A 3 , 7 = -Ai - A 2 + A 3 . 

The kernel function Kx 1 ,x 2 ,x 3 (si, S2, S3) satisfies two main properties: 

(1) K is invariant with respect to the diagonal action of ST^M). 

(2) K is homogeneous of degree — 1 — Aj in each variable Sj. 
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Hence if fj are homogeneous functions of degree — 1 + Xj, then the function 

F(s u s 2 ,s 3 ) = fi(s 1 )f 2 (s2)f 3 (s 3 )K XlMM (s 1 ,S2,s 3 ) , 
is homogeneous of degree —2 in each variable Sj € M 2 \ 0. 

5.1.2. Functional l mod . To define the model trilinear functional l mod we notice that on the 
space V of functions of homogeneous degree —2 on R 2 \ there exists a natural SX 2 (R)- 
invariant functional £ : V —>■ C . It is given by the formula £(/) = J E /<i<7 where the 
integral is taken over any closed curve £ C M 2 \ which goes around and the measure 
da on £ is given by the area element inside of E divided by 7r; this last normalization 
factor is chosen so that = 1 for the standard quadratic form Q on I 2 . 

Applying £ separately to each variable Sj G M. 2 \ of the function F(si, s 2 , s 3 ) above we 
obtain the G-invariant functional 

(18) ® /2 ® /a) := (£ ® £ ® £, F> ■ 

In the circle model this functional is expressed by the following integral: 

(19) d,.a(/i ® /2 ® /s) = ( 27T )" 3 // / fi(x)f2{y)f3(z)K XuX2M (x,y,z)dxdydz, 
where x, 2; are the standard angular parameters on the circle. 

Remark. The integral defining the trilinear functional is often divergent and the func- 
tional should be defined using regularization of this integral. There are standard proce- 
dures how to make such a regularization (see e.g. |Glj ). 

Fortunately in the case of unitary representations all integrals converge absolutely so we 
will not discuss the regularization procedure. 



5.2. Construction of test functionals. In this section we will present a construction 
of a family of test functionals pt on the space TC(E). 

Fix smooth irreducible pre-unitary representations of class one V = V T ,V = V T > and 
denote by E the smooth completion of V<g>V as in section HJ We will do the computations 
only for representations of the principal series; complementary series are treated similarly. 

For computations we will identify the spaces V and V with c°°(S l ) even . 

Our aim is to prove the following 

Proposition. There exist constants T , C, c > such that for any T >T Q there exists 
a positive functional p on Ti(E) satisfying 

(20) p(H A ) < CT 2 , 

(21) h T (X) := p(#r d ) > c for any |A| < 2T . 
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The functional pr = c 1 p is the required test functional in 14.61 

5.3. Proof of Proposition 15. 2L We will construct a functional p as an integral of ele- 
mentary functionals. Namely we find a positive function h G H(G xG) C C°°(G x G) and 
a vector u G E and define p(H) = p u (U(h)(H)), where p u is the elementary functional on 
the space 7i(E) corresponding to the vector u. 

5.3.1. Construction of function h. We construct the function h independent of parameter 
T. Let D 1 C SL 2 (M) C G be the subset of matrices g with \\g\\ < 2. We consider the 
subset D = D 1 xD 1 cGxG and choose a positive function h G H(G x G) — C™(G x G) 
which is > 1 on the subset D and is supported in some neighborhood of D. We also 
assume that the function h is invariant under left and right translations by elements of 
the maximal compact subgroup K x K. 

5.3.2. Construction of vector u. Let us identify the space E = V <S> V with the space of 
smooth functions C 00 ^ 1 x S 1 )^ 11 . Let S be a disc in S 1 x S 1 of radius (100T)" 1 . We 
construct u as a smooth non-negative real valued function on S 1 x S 1 supported in S such 
that 

(i) J udxdy = 1, 

(ii) \\u\\ 2 L2 < 10 5 T 2 . 

We would like to show that the functional p constructed in 15.31 satisfies conditions formu- 
lated in Proposition 15.21 

5.3.3. Geometric bound. We have 

p(H A ) = p u (U(h)(H A )) < C' Pu {P E ) = C'P E {u) < CT 2 

see El [EH 

5.3.4. Spectral bound. First we would like to give another description of the Hermitian 
form HV 1 ^. Consider the model trilinear functional I = l" 1 !^ described in 15.11 and the 
corresponding operator T l : E — > V\. 

We will identify the space V\ with the space c co {S 1 ) even . Fix some point z G S 1 . Then 
the 5 functional 5 Z at this point defines a functional / on E. We denote by Pf the 
corresponding Hermitian form on E, Pf(u) = \ (f,u)\ 2 . 

Since the scalar product on the space V\ ~ c°° (S 1 )^ 11 is given by the standard integral 
we see that the standard Hermitian form Py x is an average over the compact group K of 
the forms Tr(k)(P s J. This implies that Hf od = f K (Jl(k,k)(P f ))dk. 

Since we assumed the function h G H(G x G) to be K x i^-invariant we see that 
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Thus we see that in order to prove a lower bound for p(H™ od ) it is enough to establish a 
lower bound for p u (H(g)Pf) := \ (11(g) f, u) | 2 , for a subset of g G D of a measure bounded 
from below by a constant. 

Namely, the desired lower bound follows from the following 

Lemma. Let To be large enough. Then there exists an open non-empty subset D$ C D 
such that for T > Tq and for g G Dq we have \ (U(g)f,u) \ > 1/2. 

Proof. Let x, y be parameters on circles S 1 describing V and V. As follows from the 
definition (fTT?|) the functional / is given by the function / = f(x,y) on S 1 x S 1 described 
by 

f(x,y) = \sin(y - z)\ {a - 1)/2 \sm(x - z)\^- 1)/2 \sm(x - y)\^- 1)/2 , 
where a, (3, 7 G iR. 

Let D C -D be the subset of elements g G D such that the restriction of n(^)(/) to the 
subset 5 C S 1 x S 1 has the absolute value < 10. (Note that the absolute value \U(g)(f)\ 
is bounded from below for any g G D by a constant depending only onD.) It is easy to 
see that for large T the set D is a non empty subset of D C G x G of a measure bounded 
from below by a constant which is independent of T. 

On the other hand, for g G Dq we see that the gradient of the function U(g)(f) on the 
subset S is bounded by 3T. We note now that the diameter of S is bounded by (100T) -1 
and hence the lower bound on \(U(g)f,u)\ for g G -Do follows from the following easy 
claim 

Claim. Let S be a set with a measure v and u, h be two measurable functions on S . Let 
us assume that 

(i) u is real valued positive function and J udv = 1. 

(ii) sup\h(s)\ > 1 and the variation Var(h) := sup\h(s) — h(s')\ is bounded by 1/2. 
Then \ J hudv\ > 1/2. 

□ 

5.4. Construction of test functionals via Sobolev norms. In this section we outline 
another, slightly more conceptual, construction of test functionals. This construction uses 
the notion of Sobolev norms on representation spaces (see |BR2p . 

5.4.1. Sobolev norms. Let G be a Lie group and (ir, G, V) a smooth pre-unitary represen- 
tation. Then we can construct a family of positive definite Hermitian forms on the space 
V as follows. 
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Fix a basis {Xj\j = 1, ...,r} of the Lie algebra q of the group G. Then for any natural 
number I and any T > we define a Hermitian form Qi t on V by 

Q UT {v) = Y,T 2(l ~^P{X^v)) . 

V 

Here the sum is over all multi indexes v = (ni, n r ) with the norm \v\ := X] n j bounded 
by / and P = Py is the Hermitian form defining the unitary structure on V. 



5.4.2. Positive junctionals defined by forms. Every positive definite Hermitian form Q on 
V defines a positive functional pq on 7i(V) by Pq(H) = tr(H\Q). Here tr(H\Q) denotes 
the relative trace of forms H and Q; by definition it is equal to the square of the Hilbert- 
Schmidt norm of the identity operator on V considered as a morphism of pre-Hilbert 
spaces (V,Q) — > (V,H). This notion is discussed in detail in [BR2J. 



5.4.3. Construction of Sobolev test functionals. Let us apply these constructions to the 
representation (II, G x G,E) discussed in|U 

Fix I and T, consider the Sobolev Hermitian form Q = Q^t on the space E and define 
the positive functional p on TC(E) to be p = pg. 

Proposition. Suppose I > 2. Then 

(i) P (H A ) < CI*" 58 , 

(ii) There exists c> such that p(Hf od ) > cT~ 21 for\X\<2T. 
This gives another proof of Proposition 14.61 



5.4.4. Sketch of the proof of Proposition 5.4-3[ (i) Since the representation n is continu- 
ous with respect to the form Q [ T the second basic inequality 14.41 implies that p(H^) < 
C'p(Pe). The proof of the inequality p(P E ) < C"T 2 - 21 is the same as m [BR2], section 4. 

In order to prove (ii) it is enough to find a vector u G E such that Qi } t{u) < T 21 and 
I (/, u) | > c, where / = f z is the function described in 15.3.41 We can take a function 
u G C°°(5' 1 x S 1 ) of the form u = <pf where is a smooth cut-off function which equals 
around singularities of the function /. 

We leave details to the reader. 



Appendix A. 

A.l. Computation of l mod for f^-fixed vectors. In this appendix we prove the Propo- 
sition EIH which describes the assymptotic behavior of the function k\. 

One can prove this proposition applying the stationary phase method directly to the 
integral (JTHJ). To do this we need to consider the complexification of the functions e\(si) 
and the function K Xl) x 2: x 3 (si, s 2 , s 3 ) in the variables and move contour of integration 
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towards the singularities of the complexified integral. This could be done either in a 
classical language or using analytic continuation of representations in the spirit of jBRd, • 



A. 2. Computation of the integral. We prefer to prove this proposition in a different 
way. Namely we explicitly compute the value of the model functional on the unit vectors 
in terms of T- functions and then prove the proposition by applying Stirling formulas for 
assymptotic behavior of T-functions. 

Let 7T\ i , % = 1,2,3 be three representations of the generalized principal series and e Xi 
be the corresponding .fT-fixed unit vectors (they correspond to function 1 in the circle 
model). Set A(Xx, A 2 , A 3 ) := Z^^K ® e x . 2 <g> e Aa ). 

In sections IA.41 IA.5l we explicitly compute the function A(Ai, A 2 , A 3 ) (see the final expres- 
sion in IA.5|) . 

A. 3. Gaussian. We would like to compute our integral by comparing it with Gaussian 
integrals which are much easier to manipulate with. 

Namely, suppose we are given a finite-dimensional Euclidean vector space L. In this case 
we introduce the Gaussian probability measure G on L by dG = n - dimL ^ exp(-Q)dZ, 
where Q is the quadratic form which defines the Euclidean structure on L and dl is the 
standard Euclidean measure on L. 

We are interested in the quantities (f,G) = (f,G)z '■= J fdG for various (usually homo- 
geneous) functions / on L. The main properties of the Gaussian which we use are the 
following: 

(i) Normalization. (1,G) =1. 

(ii) Product formula. Suppose that the Euclidean space L is a product of Euclidean 
spaces Li and L 2 . Then the Gaussian measure G on L is the product of Gaussian measures 
G\ and G2 on Li and L 2 . In particular, if a function / decomposes as a product of 
functions fx and fa on L\ and L 2 we have (/, G) = (fx, Gx)(f2, G 2 ). 

The following integrals are classical 

Proposition. 

Let L = M. n be the standard Euclidean space. 

(i) Let r denote the radius function on L. Then (r s ,G) = T((s + n)/2)/T(n/2). 

(ii) Let h be a linear functional on L. Then (\h\ s , G) = \ \h\ \ s T((s + l)/2)/r(l/2). 

(Hi) Let L be the space M 2)2 of 2 x 2 matrices with the standard Euclidean structure. 
Then (\det\ s , G) = T((s + l)/2)r(s/2 + l)/r(l/2). 
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Proof. In (i) passing to spherical coordinates we get the integral 

2c I r s+n ^exp{-r 2 )dr = c [ u {s+n)/2 exp{-u)du/u = cT((s + n)/2) 



The normalization at s = defines the constant c = l/T(n/2). 

The proof of (ii) is reduced to the one variable case using product formula and then it 
follows from (i). 

In (iii) we can write L as a product of two column spaces L\ and L 2 . Then we have 

(\det\>,G) = J\cu(x,y)\ s dG 1 (x)dG 2 (y) = /(/ \u(x,y)\ a dG 1 (x))dG 2 (y) = 

T((s + l)/2)/r(l/2) • / \y\ s dG 2 (y) = T((s + l)/2)r(s/2 + l)/r(l/2) 

since T(l) = 1. □ 

A. 4. Reduction 1. The Proposition IA. 31 allows us to write the integrals which we would 
like to compute as some Gaussian integrals. 

Corollary. For any function h G we have 

(h,G)=T((l-X)/2)-£(h-e x ) . 

Indeed, after averaging h with respect to the action of SO (2) we can assume that it is 
proportional to the function e_\. Then the formula follows from IA.3TT j. 

Using this corollary we can rewrite the integral for the function A(Xx, X 2 , A3). 

Proposition. Consider the Euclidean space L = M 2 x IR 2 x R 2 and define the function 
B(Xi,X 2 , A3) by Gaussian integral B(Xi, X 2 , A3) := (K\ lt \ 2t \ 3 (si, s 2 , S3), G). Then 

B(Xi, X 2 , A 3 ) = A(A 1; A 2 , A 3 ) • T((l - Ai)/2)r((l - A 2 )/2)r((l - A 3 )/2) . 



A. 5. Reduction 2. Let us rewrite the integral defining the function B. First, we identify 
the Euclidean space L in section I A. 41 with the space M 2 ^ of 2 x 3 matrices. We consider 
the Euclidean space W ~ M 3 and define the map v : M 2 $ — > W using 2x2 minors. Let 
us define the function / on W by formula 

f(w u w 2 ,w 3 ) = i^i^- 1 )/ 2 !^!^- 1 )/ 2 !^!^- 1 )/ 2 . 

We can write i^Ai,A 2 ,A 3 = v*{f) and hence B(X\, X 2 , A3) = (v*(f), G) 

(here a = Ai - X 2 - A 3 , (3 = — A x + A 2 - A 3 ,7 = — A x - A 2 + A 3 as in l5.1.1j) . 

Now we will use the following general lemma which we prove in section IA.6I 

Lemma. Let h be a homogeneous function on the space W of homogeneous degree s. 
Then 

(u*(h),G) L = (h,G) w -T(s/2 + l) . 



20 



JOSEPH BERNSTEIN AND ANDRE REZNIKOV 



From this lemma we see that the computation of the function -B(Ai, A2, A3) is reduced to 
the computation of the function C(a, (3, 7) := (/, G). 

Since the Gaussian G on W is a direct product of three one dimensional Gaussians and 
the function / is a product of functions depending only on one coordinate we deduce that 
the integral (/, G) is a product of three one dimensional integrals which can be computed 
using IA.31 

Thus we obtain C(a, (3, 7) = T((a + l)/4)r((/3 + l)/4)r(( 7 + l)/4)/r(l/2) 3 . 
The final expression for the function A(Xi, A2, A3) is 



where a = X 1 - A 2 - A3, (3 = — Ai + A 2 - A 3 , 7 = — A x - A 2 + A 3 , 5 = — Ai — A 2 — A 3 . 

A. 6. Proof of Lemma IA.51 Consider the natural actions of the group 50(3) on the 
Euclidean spaces M 2i3 ps W x W and W; these actions preserve Gaussian measures. 

The map v : M 2j 3 — > W is SO(3)-equivariant; it is nothing else than the exterior product 
map W x W — > f\ (W) = W* = W. Hence we can replace the function h by its average 
with respect to the action of the group 5*0(3), i.e. up to some constant by a function 
h = r s . This shows that (u*(h), G) = a(s)(h, G), where a(s) depends on s but not on h. 

In order to compute the function a(s) we can consider the identity above for the function 
h(w) = \w 3 \ s . According to Proposition [Phi) we have (h, G) = F((s + l)/2)/r(l/2). 

On the other hand it is clear that the function v*(h) depends only on four variables and 
hence the integral (u*(h),G) coincides with the integral (h',G) over the space M 22 of 
2x2 matrices , where h'(m) = \ det(m)| s . 

From Proposition IA.3r iii) we deduce that a(s) = T(s/2 + 1). 

A. 7. Proof of Proposition According to Stirling formulas for any fixed a and large 



t we have T(cr + it) = \/2^ : exp(-||f|)|t| <J - 1 /2(i + Odt^ 1 )). 

This and the explicit formula for the function A(X±, A 2 , A3) implies the proposition. 
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